Gaze estimation is the fundamental basis for many visual tasks. Yet, the high cost of acquiring gaze datasets with 3D annotations hinders the optimization and application of gaze estimation models. In this work, we propose a novel Head-Eye redirection parametric model based on Neural Radiance Field, which allows dense gaze data generation with view consistency and accurate gaze direction. Moreover, our head-eye redirection parametric model can decouple the face and eyes for separate neural rendering, so it can achieve the purpose of separately controlling the attributes of the face, identity, illumination, and eye gaze direction. Thus diverse 3D-aware gaze datasets could be obtained by manipulating the latent code belonging to different face attributions in an unsupervised manner. Extensive experiments on several benchmarks demonstrate the effectiveness of our method in domain generalization and domain adaptation for gaze estimation tasks.

The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.

Federated embodied agent learning protects the data privacy of individual visual environments by keeping data locally at each client (the individual environment) during training. However, since the local data is inaccessible to the server under federated learning, attackers may easily poison the training data of the local client to build a backdoor in the agent without notice. Deploying such an agent raises the risk of potential harm to humans, as the attackers may easily navigate and control the agent as they wish via the backdoor. Towards Byzantine-robust federated embodied agent learning, in this paper, we study the attack and defense for the task of vision-and-language navigation (VLN), where the agent is required to follow natural language instructions to navigate indoor environments. First, we introduce a simple but effective attack strategy, Navigation as Wish (NAW), in which the malicious client manipulates local trajectory data to implant a backdoor into the global model. Results on two VLN datasets (R2R and RxR) show that NAW can easily navigate the deployed VLN agent regardless of the language instruction, without affecting its performance on normal test sets. Then, we propose a new Prompt-Based Aggregation (PBA) to defend against the NAW attack in federated VLN, which provides the server with a ''prompt'' of the vision-and-language alignment variance between the benign and malicious clients so that they can be distinguished during training. We validate the effectiveness of the PBA method on protecting the global model from the NAW attack, which outperforms other state-of-the-art defense methods by a large margin in the defense metrics on R2R and RxR.

Federated学习（FL）最近作为一种增强隐私的工具而受到了极大的关注，可以由多个参与者共同培训机器学习模型。FL的先前工作主要研究了如何在模型培训期间保护标签隐私。但是，FL中的模型评估也可能导致私人标签信息的潜在泄漏。在这项工作中，我们提出了一种评估算法，该算法可以准确计算使用FL中的标签差异隐私（DP）时，可以准确计算广泛使用的AUC（曲线下）度量。通过广泛的实验，我们显示我们的算法可以计算与地面真相相比的准确AUC。

评估对象图像的模糊对于提高对象识别和检索的性能至关重要。主要挑战在于缺乏具有可靠标签和有效学习策略的丰富图像。当前的数据集标记为有限且混乱的质量水平。为了克服这一限制，我们建议将成对图像之间的等级关系标记，而不是它们的质量水平，因为人类更容易标记，并建立具有可靠标签的大规模逼真的面部图像模糊评估数据集。基于此数据集，我们提出了一种仅以成对等级标签作为监督的方法来获得模糊分数。此外，为了进一步提高绩效，我们提出了一种基于四倍体排名一致性的自制方法，以更有效地利用未标记的数据。受监督和自我监督的方法构成了最终的半监督学习框架，可以端对端训练。实验结果证明了我们方法的有效性。

香草无监督的域适应方法倾向于用固定的神经体系结构优化模型，在现实世界中，这不是很实际的，因为目标数据通常由不同的资源有限的设备处理。因此，促进各种设备的建筑适应非常必要。在本文中，我们引入了一个简单的框架，可靠的域名适应，以通过重量分担模型库来改善跨域的概括，从中可以采样不同能力的模型，以适应不同的准确性效率折衷。该框架中的主要挑战在于同时提高模型库中众多模型的适应性。为了解决这个问题，我们开发了一种随机的集合蒸馏方法，以完全利用模型库中的互补知识进行模型间相互作用。然而，考虑到模型间相互作用与模型适应之间的优化冲突，我们将现有的BI-CLAPERIFIER域混淆体系结构扩大到优化分离的三级分类器对应物中。优化模型库后，通过我们提出的无监督性能评估指标利用体系结构的适应。在各种资源限制下，我们的框架超过了其他竞争方法，在多个基准测试方面的利润很大。还值得强调的是，即使计算复杂性降低到$ 1/64 $，我们的框架也可以保护仅源模型的性能提高。代码将在https://github.com/hikvision-research/slimda上找到。

半监督的对象检测在平均教师驱动的自我训练的发展中取得了重大进展。尽管结果有令人鼓舞，但在先前的工作中尚未完全探索标签不匹配问题，从而导致自训练期间严重确认偏见。在本文中，我们从两个不同但互补的角度（即分布级别和实例级别）提出了一个简单而有效的标签框架。对于前者，根据Monte Carlo采样，可以合理地近似来自标记数据的未标记数据的类分布。在这种弱监督提示的指导下，我们引入了一个重新分配卑鄙的老师，该老师利用自适应标签 - 分布意识到的信心阈值来生成无偏见的伪标签来推动学生学习。对于后一个，存在着跨教师模型的被忽视的标签分配歧义问题。为了解决这个问题，我们提出了一种新的标签分配机制，用于自我训练框架，即提案自我分配，该机制将学生的建议注入教师，并生成准确的伪标签，以相应地匹配学生模型中的每个建议。 MS-Coco和Pascal-VOC数据集的实验证明了我们提出的框架与其他最先进的框架相当优越。代码将在https://github.com/hikvision-research/ssod上找到。

无监督域自适应对象检测的自我训练是一项艰巨的任务，其性能在很大程度上取决于伪盒的质量。尽管结果有令人鼓舞，但先前的工作在很大程度上忽略了自训练期间伪箱的不确定性。在本文中，我们提出了一个简单而有效的框架，称为概率教师（PT），该框架旨在从逐渐发展的教师中捕获未标记的目标数据的不确定性，并以互惠互利的方式指导学生学习学生。具体而言，我们建议利用不确定性引导的一致性训练来促进分类适应和本地化适应，而不是通过精心设计的置信度阈值过滤伪盒。此外，我们与定位适应同时进行锚定适应性，因为锚被视为可学习的参数。与此框架一起，我们还提出了一种新颖的熵局灶性损失（EFL），以进一步促进不确定性引导的自我训练。配备了EFL，PT的表现优于所有以前的基线，并实现了新的最先进。

受视力语言预训练模型的显着零击概括能力的启发，我们试图利用剪辑模型的监督来减轻数据标记的负担。然而，这种监督不可避免地包含标签噪声，从而大大降低了分类模型的判别能力。在这项工作中，我们提出了Transductive Clip，这是一个新型的框架，用于学习具有从头开始的嘈杂标签的分类网络。首先，提出了一种类似的对比学习机制来减轻对伪标签的依赖并提高对嘈杂标签的耐受性。其次，合奏标签被用作伪标签更新策略，以稳定具有嘈杂标签的深神经网络的培训。该框架可以通过组合两种技术有效地从夹子模型中降低嘈杂标签的影响。多个基准数据集的实验证明了比其他最新方法的实质性改进。

卷积神经网络（CNN）通过使用大型数据集在图像分类方面取得了重大成功。但是，在小规模数据集上从头开始学习，有效地有效地学习，这仍然是巨大的挑战。借助有限的培训数据集，类别的概念将是模棱两可的，因为过度参数化的CNN倾向于简单地记住数据集，从而导致概括能力差。因此，研究如何在避免过度拟合的同时学习更多的判别性表示至关重要。由于类别的概念往往是模棱两可的，因此获取更多个人信息很重要。因此，我们提出了一个新框架，称为“吸引和修复”，由对比度正规化（CR）组成以丰富特征表示形式，对称交叉熵（SCE），以平衡不同类别的拟合和平均教师以校准标签信息。具体而言，SCE和CR学习歧视性表示，同时通过班级信息（吸引）和实例（拒绝）之间的适应性权衡缓解过度构成。之后，平均教师通过校准更准确的软伪标签来进一步提高性能。足够的实验验证了吸引和修复框架的有效性。加上其他策略，例如积极的数据增强，tencrop推断和模型结合，我们在ICCV 2021 vipriors图像分类挑战中获得了第二名。